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j  I  ABSTRACT 

A  non  candidate  constraint  in  a  linear  program  is  one  which  never 
contains  a  pivot  element  during  the  course  o£  solving  the  problem.  Dis¬ 
covering  non  candidate  constraints  is  computationally  costly  since  their 
discovery,  in  general,  depends  on  the  actual  sequence  of  pivots  used.  Know¬ 
ing  which  constraints  are  non  candidate  is  of  great  computational  benefit  since 
they  need  not  be  kept  in  updated  form.  Our  experience  indicates  that  from 
50  to  80  percent  of  the  constraints  in  randomly  problems  are  non  candidates 
at  least  part  of  the  time. 

In  this  paper  we  present  a  "learning*  approach  to  the  identification  of 
non  candidate  constraints.  At  each  iteration  we  determine  which  constraints 
can  potentially  be  pivotal;  these  are  candidate  constraints  and  all  others 
are  non  candidate  constraints  on  that  step.  On  proceeding  with  the  simplex 
method  we  update  only  the  candidate  constraints.  If  a  non  candidate  constraint 
becomes  candidate  on  a  later  step,  we  update  it  and  add  it  to  the  candidate 
list. 

Although  the  constant  checking  of  constraints  to  see  whether  they  are 
changing  from  being  candidate  to  non  candidate  is  computationally  costly,  we 
obtain  the  computational  benefit  of  having  to  keep  in  updated  form  a  much 
smaller  tableau. 


The  net  benefit  of  using  this  strategy  is  positive  and  results  in  a  25 
to  50  percent  reduction  in  total  computation  time.  This  paper  describes  the 
method  in  detail  and  gives  computational  results  of  testing  it  on  a  series  of 
randomly  generated  problems. 


THE  NON  CANDIDATE  CONSTRAINT  METHOD 
FOR  REDUCING  THE  SIZE  OF  A  LINEAR  PROGRAM 
by 

Awantl  P.  Sethi  and  Gerald  L.  Thompson 

1.  Introduction. 

Thompson,  Tonge,  and  Zionts  [4,6]  defined  a  redundant  constraint  in 
a  linear  programming  problem  as  one  which  could  be  dropped  without  changing 
the  primal  constraint  set.  Most  linear  programs  have  some  redundant  constraints, 
and  it  would  be  desirable  to  be  able  to  identify  them.  However,  the  discovery 
of  redundant  constraints  is  computationally  extremely  costly. 

A  much  weaker  concept  is  that  of  a  non-candidate  constraint,  to  be  defined 
precisely  in  Section  2.  Intuitively  a  permanently  non-candidate  constraint  is 
one  which  never  becomes  pivotal,  that  is,  never  contains  a  pivot  element,  during 
the  course  of  solving  a  given  linear  program.  Such  constraints  may,  but  need 
not,  be  redundant  in  the  strong  sense  of  [4] .  But  they  are  never  used  during  the 
course  of  solving  a  problem,  so  that  keeping  them  in  updated  form  is  of  no  value. 

The  discovery  of  permanently  non  candidate  constraints  is  computationally 
even  more  difficult  than  the  discovery  of  redundant  constraints,  since,  in 
general,  they  depend  on  the  choice  of  a  specific  sequence  of  pivots.  However, 
we  have  found  that  from  SO  to  80  percent  of  the  constraints  in  randomly  generated 
problems  are  sometimes  non  candidate  constraints,  see  Section  4,  so  that  their  dis¬ 
covery  can  be  of  great  computational  benefit. 

In  this  paper  we  present  a  "learning"  approach  to  the  identification  of 
non  candidate  constraints.  At  each  iteration  we  determine  which  constraints  can 
potentially  be  pivotal;  these  are  the  candidate  constraints;  all  others  are  non 
candidate  constraints  on  that  step.  In  proceeding  with  the  simplex  iterations 
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we  update  only  candidate  constraints,  on  the  assumption  that  non  candidate  con¬ 
straints  on  that  step  won't  become  candidates  later.  In  case  an  error  is  made, 
and  a  previously  non  candidate  becomes  a  candidate  constraint  on  a  later  itera¬ 
tion,  we  update  it,  and  add  it  to  the  candidate  list. 

I 

Although  the  constant  checking  of  constraints  to  see  if  they  are  changing 
from  being  candidate  to  non  candidate  constraints  or  vice  versa  is  costly,  there 
(  is  the  computational  benefit  of  having  a  much  smaller  tableau  to  update  at  each 

|  pivot.  The  net  benefit  of  using  this  strategy  is  positive  and  results  in  25-50 

j 

percent  reduction  in  total  computation  time9  see  Section  4* 

;  i 

[ 

2.  Basis  for  the  Method. 

The  non  candidate  constraint  method  is  merely  a  modification  of  the 
standard  simplex  method,  no  new  theoretical  results  are  needed  to  establish  the 
correctness  of  the  approach.  In  order  to  explain  it  we  set  up  some  standard 
linear  programming  terminology. 

Consider  a  linear  programming  problem  stated  form 

Maximize  {z  a  cx> 

Subject  to  Ax  «  b  (1) 

x  >_  0 

where  A  is  m*n,  b  is  mxl,  c  is  Ixn,  and  x  is  nxl.  We  assume  slack 
variables  have  been  included  in  the  definition  of  A,  and  that  the  problem  has 
been  transformed  (by  well-known  techniques)  so  that  b  >  0.  The  simplex  tableau 
at  the  kth  iteration  is  denoted  by 


A(k) 

boo 

c<k> 

z<k> 

l  /K' 

-  IV 

-  I X 


i  w,  -  .  "  * - -  .  — . .  - 

F 

*  -  —  ...  . 


(2) 


V 


where  initially 


A(0)  -  A,  b(0)  -  b,  c<°>  -  -c,  Z*0)  -  0. 


We  define  che  following  eete  of  indices 


I  ■  {1» • • • *■} 
J  *  {1* •  •  • |&} 

T  (W  _  /  A  I  J 


{j  |  j  e  J  end  Cjv  /  <  0} 


R(k>  -  U  |  1  c  I  ad 


b'k)  „<k> 

*W  «£*'>  0  *hj 


Clearly,  J'  '  Is  Che  sec  of  Indices  of  poCenClal  Incoming  variables  aC  seep 
(k) 

k,  while  Rj  Is  Che  sec  of  poCenClal  outgoing  rows  If  column  j  Is  chosen 
to  come  in. 

We  can  now  define  Che  set  of  candidate  constraints  at  Iteration  k  as 


■  {1  |  1  e  for  some  j  e  J^>  .  (8) 

The  set  of  non  candidate  constraints  at  iteration  (k)  is  then  obviously 


I  -  S' 


In  this  paper  we  will  use  the  maximum  objective  change  rule  [3,  5]  , 
which  we  will  call  the  maximum  2-change  rule.  To  find  it  we  calculate 


;(k)  _ 


Maximum 


JfcJ^.ieR^ 


Li _  ,  (k)% 

•87  1 


/"  « 


and  determine  che  indices  of  the  incoming  variable  j  and  outgoing  variable  i 
so  thac  (1)  holds  and  then 


r 
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z(fcfl)  .i(k)  +  ,(k)>  (n) 

Although  a  large  amount  of  time  is  needed  to  determine  the  pivot  giving  the 
maximum  2-change,  a  net  improvement  in  computational  performance  has  been 
found  when  it  is  used  to  solve  moderate  sized  problems  [ 5] .  In  this  paper 
we  will  show  that  still  further  improvements  can  be  obtained  by  using  the  maxi¬ 
mum  z-change  rule  in  conjunction  with  the  non  candidate  method. 

By  a  partial  pivot  for  a  given  choice  of  pivot  row  1  and  pivot  column  j, 
we  shall  mean  the  calculation  of  the  updated  right  hand  sides  tj(k+l)  from 
b  .  We  will  say  that  constraint  i  is  violated  if  b|  <  0.  Because  of 
the  choice  of  pivot  row  i  by  the  minimum  ratio  rule  (7)  and  (10),  it  is  clear 
that  no  non  candidate  constraint  at  step  k  is  ever  violated  at  step  k+1  that 
is, 

b(kH)  >  0  for  i  e  i  -  s(k)  .  (12) 

However  a  constraint  which  is  a  non  candidate  at  step  k  could  be  violated  at 
step  k+2,  k*-3,...  . 

In  the  non  candidate  method  described  in  the  next  section,  the  simplex 
tableau  is  updated  at  iteration  k  only  for  the  constraints  which  are  candidate 
at  Iteration  k.  Since  from  SO  to  80  percent  of  all  constraints  in  randomly 
generated  problems  are  non  candidate,  this  results  in  considerable  computational 
savings,  see  Section  4  . 

3.  The  Non  Candidate  Algorithm. 

The  flow  chart  for  the  non  candidate  simplex  algorithm  is  shown  in  Figure  1. 
The  initialization  steps  of  finding  the  set  S^  of  candidate  constraints  and 
the  first  maximum  z-change  pivot  are  given  in  boxes  1  and  2.  They  can  be  completed 
by  making  one  pass  through  the  original  data  of  the  problem.  Step  3  is  the  pivot 
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step  which  updates  the  rows  of  the  simplex  tableau  restricted  to  the  candidate 

constraints.  The  rows  corresponding  to  the  non  candidate  constraints  are  not 

updated.  In  box  4  a  test  is  made  to  see  if  the  current  tableau  is  optimal;  if 

it  is  the  algorithm  stops;  otherwise  it  continues. 

Is  box  5  of  Figure  1,  the  computer  is  instructed  to  make  one  pass  through 

(k) 

the  updated  tableau  at  the  kth  step  which  is  restricted  to  the  rows  in  S  ; 

(k) 

during  the  data  pass  all  non  candidate  constraints  are  removed  from  S  to 
form  the  set  During  this  data  pass  it  is  possible  to  also  carry  out 

the  instruction  in  box  6  of  finding  the  maximum  z-change  pivot  in  the  tableau 
restricted  to  s^k+l) . 

(k) 

In  box  7,  the  computer  is  instructed  to  perform  a  partial  pivot  on  b 

restricted  to  the  set  g^+l).  at  the  end  of  the  partial  pivot  b  restricted 

to  the  same  set  and  hence  x^^*^  will  be  available.  As  instructed  in  box  8, 

the  proposed  solution,  x^^,  is  then  substituted  into  the  non  candidate 

(k) 

constraints  in  the  set  I  -  S  to  see  if  any  of  them  is  violated.  If  any 
such  constraint  is  found  to  be  violated,  the  computer  is  instructed  in  box  9 
to  update  that  constraint,  and  then  go  to  box  5  to  try  again  to  find  a  maximum 
z-pivot  and  a  new  This  process  is  repeated  until  eventually  a  proposed 

x(k*l)  will  be  found  which  does  not  violate  any  non  candidate  constraint.  When 
this  happens,  the  computer  goes  from  box  8  to  box  10  where  k  is  updated;  and 
from  there  it  goes  back  to  box  3  to  continue  with  the  pivoting  process. 

Since  the  algorithm  outlined  is  simply  a  reorganization  of  the  standard 
simplex  method,  all  of  the  anti-cycling  techniques,  and  finiteness  proofs  for 
the  latter  hold  for  the  non  candidate  method  as  well. 

4.  Computational  Experience. 


Table  1  gives  the  time  comparison  between  the  non  candidate  greatest 
z-change  method  and  the  greatest  z-change  method  in  solving  a  series  of  19 
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randomly  generated  problems  with  two  different  densities  of  problem  data. 

Since  the  same  entry  criterion  was  used  In  each  method  the  sequence  of  pivots 
was  Identical  In  each.  Hence  the  differences  in  solution  times  can  be  attributed 
entirely  to  the  relative  efficiencies  of  the  two  methods.  Ve  found  that  the  non 
candidate  method  is  more  effective  on  problems  with  higher  density,  and  also  on 
larger  problems. 

Table  2  gives  a  computational  comparison  of  the  non  candidate  method  and 
the  ordinary  simplex  method  on  two  sets  of  randomly  genr  ated  problems.  Note 
that  the  number  of  iterations  is  smaller  for  the  non  candidate  method  due 
to  Its  greatest  z-change  entry  criterion.  The  ratio  of  times  for  the  two  methods 
are  close  to  the  ratio  of  number  of  iterations,  which  indicates  that  the  savings 
In  time  from  not  updating  non  candidate  constraints  approximates  the  additional 
time  needed  for  the  greatest  z-change  calculations. 

We  have  so  far  not  tried  to  compare  the  two  methods  on  actual  problems 
derived  from  real  applications.  However,  the  results  so  far  on  these  randomly 
generated  problems  are  very  encouraging. 

5.  Variations  and  Extensions  of  the  Method. 

Many  variants  of  the  method  described  In  this  paper  have  occurred  to  us 
which  we  have  not  yet  had  time  to  test.  We  discuss  some  of  them  here. 

(a)  Connections  with  the  GUB  Method.  The  generalized  upper  bound  (GUB)  [6] 
method  handles  constraints  which  are  sums  of  variables  outside  of  the  tableau. 
Usually  such  GUB  constraints  are  also  non  candidate  in  our  sense.  When  they  are 
candidate  constraints  and  are  actually  chosen  to  be  pivotal,  they  are  not  re¬ 
introduced  into  the  tableau  as  we  do;  Instead  special  pivoting  rules  are  added 
to  the  simplex  algorithm.  Perhaps  such  special  pivoting  rules  could  be  extended 
to  more  general  kinds  of  constraints. 
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(b)  Intermittent  candidate  checking.  Instead  of  checking  for  candidate 
constraints  at  each  iteration  as  we  do  in  box  5  of  Figure  1,  we  could  instead, 
perform  this  check  only  once  every  sth  iteration,  s  *  5,  say,  where  the  danger 
is  that  a  constraint  could  then  become  violated,  and  dual  feasibility  lost.  This 
could  be  corrected  by  performing  some  dual  pivots.  In  this  way  the  method  could 
become  a  kind  of  primal-dual  algorithm.  Only  actual  computational  tests  could 
show  whether,  and  for  what  values  of  s,  this  method  would  be  good. 

(c)  Extension  to  large  problems.  Most  large  linear  programming  problems 
are  very  sparse  initially,  but  the  tableau  can  fill  in  during  the  course  of  the 
computation.  Hence  the  non  candidate  method  is  a  potentially  valuable  way  of 
keeping  down  the  size  of  the  working  tableau,  and/or  the  size  of  the  basis  inverse 
matrix.  The  difficulty  is  in  preforming  the  check  for  candidate  constraints. 

To  this  end  we  propose  that  the  original  data  be  stored  on  two  tapes,  once  in 
row  order,  and  once  in  column  order.  Then  the  tape  with  row  ordered  data  is 
used  to  check  for  candidates,  and  the  tape  with  column  ordered  data  used  to  check 
for  incoming  variables.  We  believe  that  the  computations  could  be  organized  so 
that  both  of  these  steps  could  be  done  efficiently.  If  the  number  of  candidate 
constraints  is  as  small  as  we  found  in  Table  2,  then  dramatic  improvements  in 
the  solution  of  large  linear  programs  might  be  possible  using  this  method. 

6.  Conclusions . 

In  this  paper  we  have  defined  the  concept  of  candidate  and  non  candidate 
constraints,  and  have  demonstrated  computationally  that  a  modification  of  the 
simplex  method  can  be  made  to  take  advantage  of  the  fact  that  non  candidate 
constraints  need  not  be  updated.  The  method  assumes  that  a  non  candidate  con¬ 
straints  at  step  k  will  remain  so.  If  an  error  is  made  and  such  a  constraint 
becomes  a  candidate  at  a  later  step  then  the  constraint  is  updated  and  put  back 


f 
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Into  the  tableau.  Significant  computational  savings  are  possible  by  using  this 
method.  Extensions  of  the  method  to  find  the  solution  to  large  problems  were 
sketched. 
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Figure  1.  Flow  chart  for  non  candidate  simplex  algorithm. 
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Table  1 

Comparison  of  greatest  z-change  non  candidate 
and  simplex  methods.  The  sequence  of 
pivots  was  the  same  for  both  methods,  so 
that  the  ratio  of  times  indicates  the 
relative  efficiency  of  the  two  methods. 
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Problem  Sec  1.  Coefficients  were  generated  at  random  with  50%  positive, 

10%  negative,  and  40%  zero. 
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Problem  Set  2.  Coefficients  were  generated  at  random  with  65%  positive, 

15%  negative,  and  20%  zero. 


Table  2 

Comparison  of  the  greatest  z-change  non  candidate  method 
and  the  simplex  method  with  most  negative  reduced 
cost  entry  criterion. 


-12- 


References 


[1] .  Charnes,  A.,  "Structured  Sensitivity  Analysis  in  Linear  Programming  and  an 

Exact  Product  Form  of  Inverse,"  Naval  Research  Logistic  Quarterly. 

15  (1958),  517-522. 

[2] .  Jeroslov,  "The  Simplex  Algorithm  with  the  Pivot  Rule  of  Maximizing  Criterion 

Improvement,"  Discrete  Mathematics,  4  (1973),  367-377. 

[3] .  Lemke,  C. ,  "The  Dual  Method  of  Solving  the  Linear  Programming  Problems," 

Naval  Research  Logistics  Quarterly.  1  (1954),  36-47- 

[4] .  Thompson,  G.  L. ,  S.  Tonge  and  S.  Zionts,  "Techniques  for  Removing  Non- 

Binding  Constraints  and  Extraneous  Variables  from  Linear  Programming 
Problems,"  Management  Science.  12,  (1966),  588-608. 

[5] .  Wolfe,  P.  and  L.  Cutler,  "Experiments  in  Linear  Programs  (Run  41)"  in  the 

book  Recent  Advances  in  Mathematical  Programming,  edited  by  R.  L. 
Graves  and  P.  Wolfe,  New  York,  McGraw  Hill,  1963. 

[6] .  Zionts,  S.,  Linear  and  Integer  Programming.  Prentice  Hall,  New  Jersey,  1974. 


Unclassified 

»y.umii  .u»»lHv.«iigHO>  Tms  mat,*  ;mnm  Uma  amiatwu) 

REPORT  DOCUMENTATION  PAGE 


n  numMA - 

i  MSRR  SO.  455 


,a.  oovt  accession 


mad  oirrBucnoNS 

SgFOKE  COMW.STIWC  ronu 

siSfwiSPi  ^SVSSSFSiuSSBS  " 


«•  rtrue  SMfaMMi 


THE  NON  CANDIDATE  CONSTRAINT  METHOD  FOR 
REDUCING  THE  SIZE  OF  A  LINEAR  PROGRAM 


j7.  Mj  ThoH'l)  '  ' 

j  Awanti  P.  Sethi 

i  G.  L.  Thompson 

aentonmiks  oaoanizatTSn  name  and  aooaeu 
;  Graduate  School  of  Industrial  Administration 

{  Carnegie-Mellon  University 

!  Pittsburgh,  Pennsylvania  15213 

it*-  orncc  Inc  acouzss 

(  Personnel  and  Training  Research  Programs 

Office  of  Naval  Research  (Code  458) 

L, - Arlington,.  Virginia  _ 

t  '*■  monitoniho  agency  namc  •  md*mih  SiES  mm  Emm mSg  oSm 


I.  TYNE  or  NKSONT  A  AEMOO  COVCMCO 

Technical  Report 
February  1983 _ 


«.  CONTNAC 


II AMT  MUMSCWu, 


N00014-75-C-0621 


UL-'^NoaAAM  element,  awo-iect,  taia 

^  ANSA  A  WOMK  UNIT  NUMOENS 

NR  047-048 

12.  ASSORT  OATS 

February  1980 
<)■  MUMOEA  or  AAeeS 

11 _ 

II.  iccumry  CLASS,  (at  I Ms 


<  IS.  OrJTWlSU'nON  STATEMENT  ,0/  tS!m  Hapatt) 


•i*.  okcl.mim*icat:o'j/u5wnc^au..'< j 
SCHEDULE 


Approved  for  release;  distribution  unlimited. 


.  17.  OISTAiauTtON  STATEMENT  <al  (.*>•  1 


1  iMmS  M  JIM  29,  It  < 


;  is.  suaalcmemtaav  notes 


i  IS.  KEY  WOAOS  (Camunmm  am  rararaa  1 


•  «.  iawis  *r  MmA  1 


|  simplex  method  j 

5  non  candidate  constraint  simplex  method  ( 

20.  ABSTRACT  rCamttima  am  raaaraa  H0a  If  naaaaiarr  m*  imanttfr  ay  Mss A  mmmmaa)  I 

A  non  candidate  constraint  in  a  linear _program  is  one  which  never  contains! 
a  pivot  element  during  the  course  of  solving  the  problem.  Discovering  non  > 
candidate  constraints  is  computationally  costly  since  their  discovery,  in  gen- 3 
eral,  depends  on  the  actual  sequence  of  pivots  used.  Knowing  which  constraints 
are  non  candidate  is  of  great  computational  benefit  since  they  need  not  be  kept 
in  updated  form.  Our  experience  Indicates  that  from  50  to  80  percent  of  the 
<  constraints  in  randomly  problems  are  non  candidates  at  least  part  of  the  time. 
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In  this  paper  ve  present  a  "learning"  approach  to  the  Identification  of 
non  candidate  constraints.  At  each  Iteration  we  determine  which  constraints 
can  potentially  be  pivotal;  these  are  candidate  constraints  and  all  others  are 
non  candidate  constraints  on  that  step.  On  proceeding  with  the  simplex  method 
we  update  only  the  candidate  constraints.  If  a  non  candidate  constraint  become 
candidate  on  a  later  step,  we  update  it  and  add  It  to  the  candidate  list. 


Although  the  constant  checking  of  constraints  to  see  whether  they  are 
changing  from  being  candidate  to  non  candidate  is  computationally  costly,  we  j 
obtain  the  computational  benefit  of  having  to  keep  in  updated  form  a  touch 
smaller  tableau. 

The  net  benefit  of  using  this  strategy  Is  positive  and  results  In  a 
25  to  SO  percent  rduction  In  total  computation  time.  This  paper  deecrlbes  the 
method  In  detail  and  gives  computational  results  of  testing  it  on  a  series  of 
randomly  generated  problems. 


